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SUMMARY 

Over the past several decades, federal and state governments, school districts and education 
leaders have invested significant resources to develop assessments and data systems to track 
student performance. Yet little attention has been paid to the importance of building the capacity 
of teachers in assessment so that they are prepared to use data on student learning to inform 
and improve instruction. As part of our National Review of Teacher Preparation Programs, 
the National Council on Teacher Quality (NCTQ) is examining the extent to which teacher 
preparation programs train teachers to understand assessment data and use that data to make 
informed instructional adjustments. Our preliminary findings are cause for concern: At best, 
teacher preparation programs in this preliminary sample are providing limited training to candidates 
in the field of assessment and data use. As a result, schools end up employing teachers who lack 
the training necessary to make use of information increasingly at hand. 

BACKGROUND 

Spurred by state and federal accountability systems, an increasing number of the nation’s school 
districts are working to implement “data driven instruction.” Data driven instruction, though lacking 
a formal definition, has come to represent an organized and collaborative commitment by educators 
to develop or adjust instruction based on a broad set of data on student learning, demographics, 
attendance and school climate. The key question in schools and classrooms across the country 
has shifted from “Was the content taught?” to “How much have our students learned?” and then 
“How can we help our students understand what they haven’t learned?” 

Federal and state governments have spent millions of dollars creating the infrastructure that data 
driven instruction demands. Over $500 million of federal funding has been spent on developing 
states’ technology infrastructure to support data driven decision-making through the Statewide 




Longitudinal Data System (SLDS) program.' Secretary of 
Education Arne Duncan made data driven decision-making 
a national priority by requiring recent Race to the Top 
investments to address improving data systems and creating 
high quality assessments. Among organizations, the Data 
Quality Campaign has been instrumental in marshalling 
efforts to build educators’ data literacy.^ 

But ask teachers, especially new teachers, about their 
comfort level with applying data in order to make 
instructional decisions and you’re likely to get the same 
answer: They are not at all comfortable with much of 
anything about the process. 

Reports from those districts that have implemented 
data driven instruction and made big gains in student 
performance indicate that developing teacher capacity in 
the process can take years. ^ Accordingly, it is incumbent 
upon initial teacher preparation programs to lay the proper 
foundation for teacher candidates’ understanding of data 
driven instruction. This basic training should focus on 
data that is derived from assessments of student learning, 
with “assessment” broadly construed to include a range 
from informal checks of understanding using classroom 
exercises to formal, standardized tests. 

While other data on students and schools can be useful to 
teachers, being able to understand and analyze assessment 
data is essential. Hence our goal in this review: ascertain the 
extent to which teacher preparation programs are providing 
new teachers with the foundation in using assessment data 
that they need to succeed. To date, there has been no systematic 
examination of teachers’ preparation in assessment and 
the use of assessment data in data driven instruction. Indeed, 
no framework for conducting such a systemic examination 
has even been advanced. 

This first analysis evaluates all of the coursework related 
to assessment in a preliminary representative sample of 



48 elementary and secondary programs, housed in 29 
higher education institutions in nine states. In May 2012 
we will expand this initial sample to report on approximately 
200 programs housed in 100 institutions in 25 states. 

Our findings provide the basis for our recommendations 
for federal and state agencies, school districts and foundations 
to help them leverage improvements in assessment course- 
work in teacher preparation programs. 

What foundation must teachers have to begin to 
practice data driven instruction? 

Every day, teachers make a tremendous number of both 
short- and long-term decisions about instruction. Indeed, 
it is estimated that teachers make around 1 1 ,000 signiEcant 
instructional decisions in any given year.'' They should 
base such decisions on evidence of student mastery of 
content as revealed by any in a wide range of assessments. 

To that end, it makes sense that the first thing that a teacher 
candidate should develop is some degree of assessment 
literacy. In other words, teacher candidates should 
understand the taxonomy of assessment (distinguishing 
between “formative” and “summative,” “norm-referenced” 
and “criterion-referenced” assessments, and so on), the 
kinds of data produced by each type of assessment, and the 
uses for such data. 

Second, teacher candidates should develop the right 
analytical skills to apply to assessment data. In addition to 
assessment literacy, they need to know how to dissect, 
describe and display the data that emerges from assessments. 

Lastly, teacher candidates need to learn how to use assessment 
data to make the best decisions about what and how to 
teach and re-teach: instructional decision-making. 

Eor this memo and the larger review, we evaluated the 
extent to which required coursework in undergraduate 
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elementary and secondary programs covers these three 
critical domains of assessment: literacy, analytical skills 
and instructional decision-making. 

An important note regarding our ratings methodology: 
In order to receive the highest rating in any of the three 
domains, the program would not only have to provide 
instruction on the topic, but also provide opportunities 
for practice (in the form of class exercises, culminating 
projects, or the like). 

FINDINGS 

How adequate is program coverage of assessment 
literacy, in which the teacher learns and practices the 
taxonomy of assessment? 




Virtually Very Limited Partially Adequate 
none limited adequate 

Coverage 



■ Elementary (n=27) 

■ Secondary (n=21) 

Only 21 percent of the programs in the sample cover literacy 
topics adequately, with an additional 25 percent doing so with 
partial adequacy. More than half of all programs have very limited 
or limited coverage. 

One in five programs (21 percent) in our sample earned 
the highest score on this first domain of assessment literacy 
because their coursework provides: 1) comprehensive 
coverage of both classroom and standardized assessment 
(including key concepts such as validity and reliability), 
and 2) practice on developing and scoring assessments. 
However, in just over half of the programs (54 percent), the 
coverage came up short, usually because courses focus 
too exclusively on classroom assessments and do not build 



crucial skills in making use of standardized test results. 
Facility with classroom assessment is an important skill, 
but an inability to synthesize data from district and state 
standardized tests with data from classroom assessments 
can cripple novice teachers and impair their capacity to 
improve student performance. 

The findings are considerably bleaker in the second domain 
of analytical skills. In general, coursework appears to provide 
only the most basic tools for analysis of assessment data. 

How adequate is teacher preparation program coverage 
of analytical skills, in which teacher candidates dissect, 
describe and display the data that emerges from 
assessments? 




none limited adequate 

Coverage 

■ Elementary (n=27) 

■ Secondary (n=21) 

Only 2 percent of the programs in the sample cover analytical 
skills analysis adequately, with an additional 13 percent doing 
so with partial adequacy. The vast majority of programs (85 
percent) have no, very limited or limited coverage. 

Only four programs (8 percent) expose teacher candidates 
to the process of analyzing data from standardized assessment. 
Indeed only three of the 48 programs (6 percent) stress 
in course materials the terminology and concepts that a 
candidate would find necessary to interpret standardized 
test scores. In 2 1 programs (44 percent) , teacher candidates 
analyze classroom assessment data through independent 
“capstone projects,” but this exercise appears to be 
candidates’ only practice in three of four such programs 
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seeing as how they have no relevant assignments associated 
with coursework. Moreover, while teachers in schools 
now routinely engage in collaborative analysis, we found 
evidence of collaboration in only five preparation programs’ 
class assignments (10 percent) and one programs’ capstone 
project (2 percent). 

How adequate is teacher preparation program cover- 
age of instructional decision-making, in which teachers 
use assessment data for planning instruction? 

16 15 




none limited adequate 

Coverage 

■ Elementary (n=27) 

■ Secondary (n=21) 

No programs in the sample cover instructional decision-making 
adequately, with 1 3 percent doing so with partial adequacy. The 
vast majority of programs (87 percent) have no, very limited or 
limited coverage. 

In this third domain of instructional decision-making, the 
findings remain bleak, with no program adequately addressing 
this assessment domain. Many programs do provide at least 
cursory exposure to the concept of “formative assessment,” 
which constitutes a toehold on how to use assessment data for 
planning. However, we found that programs generally do not 
use their methods courses as the vehicles for more nuanced, 
subject-specific practice addressing, for example, common 
but subtle misconceptions and misinterpretations that might 
underhe incorrect answers, or how assessment results might 
suggest necessary instructional scaffolding. 



How adequately overall do teacher preparation 
programs cover all three domains of assessment? 

While assessment is addressed to some extent in every 
one of the 48 programs we examined, only two elementary 
and one secondary program (6 percent) provide preparation 
that can be deemed “adequate.” 

Coverage relative to this overall metric can be classified as 
“partially adequate” in another four elementary and six 
secondary programs (2 1 percent) . 

The remaining 2 1 elementary and 1 4 secondary programs 
(7 3 percent) in the sample proved to contain such substantial 
weaknesses in one or more of the three domains of 
assessment that their preparation can only be classified as 

“inadequate.” 

What is an acceptable program? Our definition has three 
parts: 

First, it provides the teacher candidate with a strong 
foundation in assessment literacy, with attention to the 
instructional role of standardized tests, particularly the 
program state’s standardized tests. 

Second it requires the teacher candidate to apply that 
knowledge, specifically by preparing formative and 
summative classroom assessments. 

Lastly, it gives the teacher candidate a relatively strong 
foundation in interpreting and applying data, both alone and 
working with a team, and does so using both standardized 
and classroom assessments. 

These three parts are based roughly on the three domains, 
but combining them into an overall metric moves us 
away from a simple aggregation of ratings in each domain. 
Frankly, this allows us to deem “acceptable” a program 
that may not have received the highest rating on one or 
both of the second and third domains, but still provides 
a “relatively strong” foundation in the competencies they 
address. 
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Overall coverage of the three assessment domains 

3 

Adequate 




Only 6 percent of programs in the sample cover assessment 
adequately, with 21 percent doing so with partial adequacy. 
Almost three-quarters of programs (73 percent) have inadequate 
coverage. 

We found that neither differences in state regulations nor 
the nature of a state’s professional standards explains any 
differences in the adequacy among programs’ coverage 
of assessment. States with explicit expectations regarding 
teachers’ knowledge of assessment were more likely to 
have an institution with better outcomes, but by no means 
did this apply to all of the institutions in the sample in any 
given state. As far as program accreditation goes, the difference 
points to a negative impact of accreditation: Programs in 
institutions that are not accredited by NCATE were more 
likely to provide adequate coverage of assessment. 

RECOMMENDATIONS 

We fully recognize that assessment and use of assessment 
data is a sufficiently large and important topic for digestion 
by teacher candidates that no matter how adequate a 
program may be in laying a foundation, continued training 
will be necessary to build and hone knowledge and skills 
during induction and throughout classroom teaching. 
But the need for in-service training does not negate the 
need for strong pre-service preparation. There are numerous 
levers that should be used to provide both increased pressure 
on and increased support for teacher preparation programs 
to lay a better foundation in this area: 



Invest in research 

There are a number of issues related to data driven instruction 
that warrant exploration. Federal policy should support 
research into the effects of data driven instruction on student 
achievement, the knowledge and skills teachers need to 
become expert at data driven instruction, the conditions 
within schools that are necessary for data driven instruction 
to have maximum impact, the collection and dissemination 
of best practices related to pre-service and in-service 
training, and the compilation and meta-analysis of existing 
research on assessment, data driven instruction, and related 
topics. 

Build teacher capacity 

The federal government should encourage better teacher 
candidate preparation in data driven instruction by: 
1) amendments to Title II of the Higher Education Act 
(HEA) to provide suitable incentives to teacher preparation 
programs to prepare candidates to understand and use 
assessments and assessment data, 2) requiring that a 
portion of SLDS funds be used for training on assessment 
and data skills, and 3) allocating a percentage of funds 
under Title II of the Elementary and Secondary Education 
Act (ESEA) for training. Amendments to HEA and ESEA 
respectively could also: 1) provide states with incentives 
to ensure that their program approval authority reinforces 
the importance of teacher data skills, and 2) encourage 
innovative state and local practices in data driven instruction. 

Increase accountability of preparation programs 

Two different and potentially complementary approaches 
might increase the accountability of preparation programs 
with regard to the quality of their assessment coursework: 

■ DISTRICTS 

Of all the stakeholders in the country’s educational 
system, school districts have the greatest vested interest 
in hiring more assessment-capable teachers. Now - of 
necessity — they must hire new teachers who no sooner 
arrive at their doorsteps certified to teach than they need 
to be provided professional development on a variety 
of assessment tools and tasks necessary for school 
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improvement. Districts that hire a large share of graduates 
from any given teacher preparation program might wish 
to use their bargaining power to bring the program 
to the table for collaborative work to ensure that its 
teacher candidates arrive with a better foundation in 
all assessment domains. 

Where such collaboration is infeasible, better screening 
of all candidates on assessment knowledge in the hiring 
process would be helpful. We recommend that the 
Council of the Great City Schools develop a test of 
knowledge in all assessment domains that can be used 
in short order for hiring purposes by school districts. 

■ STATES 

Teacher licensing tests can drive significant change in 
teacher preparation programs. One possible reason that 
assessment is addressed so poorly in teacher preparation 
programs is that it is not covered at any depth in the 
licensing tests required by most states. As a first step to 
remedying this situation, the Council of Chief State 
School Officers (CCSSO) could evaluate the scope of 
topics addressed in those licensing tests that do include 
assessment to make recommendations to states about 
how licensing tests might be improved in this area. 
If warranted, CCSSO could sponsor a consortium of 
states who wish to aggressively promote licensing test 
enhancements. 

Develop model exercises and curriculum. 

The most sophisticated coursework on data driven instruction 
offered to practicing teachers and adrninistrators requires that 
they bring to class their district’s student performance data for 
use in analytical exercises. One of the impediments to creating 
similar exercises in initial certification program coursework is 
that no one has easy access to that kind of data inventory. To 



spur the same kind of exercises for teacher candidates, states 
and foundations should develop “canned” elementary and 
secondary school data sets that contain fabricated but 
realistic student performance data from a typical range of 
classroom, district and state assessments. These data sets could 
be used by teacher educators to develop a very rich collection 
of assessment exercises designed to develop teacher candidate 
capacity in literacy, analytical skills and instructional decision- 
making in existing coursework or in model course curricula 
developed by foundations and their partners. 

CONCLUSION 

The assessment knowledge that most initial certification 
programs see as necessary for teacher candidates and the 
assessment knowledge that district and state personnel see 
as necessary for teachers are simply not the same. In too 
many programs, what assessment coursework is required 
centers only on the classroom, preparing teacher candidates 
to develop and use assessment data to improve their students’ 
performance in an insular environment. Important as 
this type of preparation may be, it short-changes teacher 
candidates because it does not simulate the environment 
in which they will work. Those candidates who are hired 
from programs with this highly circumscribed introduction 
to the three assessment domains will probably find 
themselves confronting data presentations that use terms 
and concepts to which they have never been exposed, some 
as early as their very first faculty meeting. Today’s schools 
demand teachers who can comfortably understand and 
utilize — both individually and coUaboratively — a full range 
of classroom and standardized data, whether it relates to 
their own students or aU students in their school. Preparing 
them for anything less is unfair to teacher candidates as 
well as to the many students they plan to teach. 
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